Techniques for correlating service events in computer network diagnostics

ABSTRACT

Examples described herein generally relate to receiving a query context for service events occurring on one or more networks, determining, based on the query context, a set of service events occurring on the one or more networks, querying multiple layers of a multiple-layer relational graph to determine one or more other service events having a defined relationship with the set of service events at one or more of the multiple layers, where the multiple layers include a configuration layer, an observation layer, and learned layer, defining relationships between services or service events, and indicating, via a user interface and in response to the query context, the one or more other service events.

CLAIM OF PRIORITY UNDER 35 U.S.C. §119

The present Application for Patent claims priority to ProvisionalApplication No. 62/850,300, entitled “TECHNIQUES FOR CORRELATING SERVICEEVENTS IN COMPUTER NETWORK DIAGNOSTICS” filed May 20, 2019, which isassigned to the assignee hereof and hereby expressly incorporated byreference herein for all purposes.

BACKGROUND

Large-scale networked systems are provided as platforms employed in avariety of settings for running service applications and maintainingdata for business and operational functions. Such networks can includeand/or be a part of a data center (e.g., a physical cloud computinginfrastructure) that may provide a variety of services (e.g., webapplications, email services, search engine services, resource sharingservices, etc.) for client computing devices connected to at least aportion of the network. These large-scale networked systems typicallyinclude a large number of resources distributed throughout the datacenter, where each resource can include or at least resemble a physicalmachine.

In the realm of telemetry for monitoring health of network resources, avast number (e.g., billions) of metrics are collected from or forresources over a period of time (e.g., each second) of a given network.Due to the number of metrics, it can become difficult to keep track ofthe metrics and/or related signals, health status of the networkresources, etc. In addition, when services experience issues, engineersthat maintain the services and/or corresponding resources may benotified by system alarms tens or hundreds of times, and the engineersdo not always know which alarm is the most important to respond to, ormay miss important alarms due to the sheer number of alarms, etc. Issuescan also be caused by downstream dependencies, and without the necessarydomain knowledge, it can be difficult to understand what signals areaffecting a given service, and/or how to locate/determine a dependencythat may ultimately be causing the issue.

SUMMARY

The following presents a simplified summary of one or more examples inorder to provide a basic understanding of such examples. This summary isnot an extensive overview of all contemplated examples, and is intendedto neither identify key or critical elements of all examples nordelineate the scope of any or all examples. Its sole purpose is topresent some concepts of one or more examples in a simplified form as aprelude to the more detailed description that is presented later.

In an example, a computer-implemented method for detecting relatedservice events in a network diagnostic application is provided. Themethod includes receiving a query context for service events occurringon one or more networks, determining, based on the query context, a setof service events occurring on the one or more networks, queryingmultiple layers of a multiple-layer relational graph to determine one ormore other service events having a defined relationship with the set ofservice events at one or more of the multiple layers, where the multiplelayers include a configuration layer defining configured relationshipsbetween services or service events, an observation layer definingobserved relationships between services or service events, and learnedlayer defining algorithmically-determined relationships between servicesor service events, and indicating, via a user interface and in responseto the query context, the one or more other service events.

In another example, a computing device for detecting related serviceevents in a network diagnostic application is provided that includes amemory storing one or more parameters or instructions for identifyingrelated service events from a service event repository, and at least oneprocessor coupled to the memory. The at least one processor isconfigured to receive a query context for service events occurring onone or more networks, determine, based on the query context, a set ofservice events occurring on the one or more networks, query multiplelayers of a multiple-layer relational graph to determine one or moreother service events having a defined relationship with the set ofservice events at one or more of the multiple layers, where the multiplelayers include a configuration layer defining configured relationshipsbetween services or service events, an observation layer definingobserved relationships between services or service events, and learnedlayer defining algorithmically-determined relationships between servicesor service events, and indicate, via a user interface and in response tothe query context, the one or more other service events.

In another example, a non-transitory computer-readable medium includingcode executable by one or more processors for detecting related serviceevents in a network diagnostic application is provided. The codeincludes code for receiving a query context for service events occurringon one or more networks, determining, based on the query context, a setof service events occurring on the one or more networks, queryingmultiple layers of a multiple-layer relational graph to determine one ormore other service events having a defined relationship with the set ofservice events at one or more of the multiple layers, where the multiplelayers include a configuration layer defining configured relationshipsbetween services or service events, an observation layer definingobserved relationships between services or service events, and learnedlayer defining algorithmically-determined relationships between servicesor service events, and indicating, via a user interface and in responseto the query context, the one or more other service events.

To the accomplishment of the foregoing and related ends, the one or moreexamples comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more examples. These features are indicative, however, ofbut a few of the various ways in which the principles of variousexamples may be employed, and this description is intended to includeall such examples and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an example of a computing device fordetermining relationships among service events in accordance withexamples described herein.

FIG. 2 is a flow diagram of an example of determining relationshipsamong service events in accordance with examples described herein.

FIG. 3 is a flow diagram of an example of generating a multiple-levelrelational graph of service events in accordance with examples describedherein.

FIG. 4 is a diagram of an example of a user interface of a networkdiagnostic application in accordance with examples described herein.

FIG. 5 is a graphical representation of an example of signalscorresponding to underling service events in accordance with examplesdescribed herein.

FIG. 6 is an example of a relational graph representation of a serviceand related service events in accordance with examples described herein.

FIG. 7 is a schematic diagram of an example of a computing device forperforming functions described herein.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well known components are shown in blockdiagram form in order to avoid obscuring such concepts.

Described herein are various examples related to correlating serviceevents, or underlying incident records, to facilitate determining, for agiven service event, one or more correlated service events. This canfacilitate improved service event analysis, root cause prediction, alertnoise reduction, and/or the like. For instance, a multiple-layerrelational graph can be generated and employed to define relationshipsamong service events, and the graph can be queried to determine, for agiven service or service event, the correlations and/or correspondingpatterns at one or more of the layers to determine a set of relatedservices or service events. For example, the multiple-layer relationalgraph can include a configuration layer that defines relationshipsbetween services and/or between service events based on a storedconfiguration. In addition, for example, the multiple-layer relationalgraph can include an observation layer that defines relationshipsbetween services and/or between service events based on observed networkactivity and/or usage of a network diagnostic system. Moreover, forexample, the multiple-layer relational graph can include a learned layerthat defines relationships between services and/or between serviceevents based on algorithmic determinations about the services and/orservice events (e.g., around parameters thereof).

In an example, given a query context of a service and/or service event,the multiple-layer relational graph can be queried to determine thecorrelated services and/or service events, patterns of correlationsbetween the services and/or service events, etc. at each layer todetermine other services and/or service events that are possibly ofinterest (e.g., that have some correlation). In one example, thecorrelations or related metrics can be weighted at each layer based onthe layer itself (e.g., to assign different weights in general toconfigured, observed, learned, etc. correlations) and/or based on otherparameters regarding the correlation. Where the correlation or relatedmetric achieves a threshold, in one example, the corresponding serviceand/or service event may be indicated for the query context to identifypossibly related services and/or service events. This can assist inreducing the number of services and/or service events to be observed indiagnosing the service or service event that is the subject of the querycontext.

Turning now to FIGS. 1-7, examples are depicted with reference to one ormore components and one or more methods that may perform the actions oroperations described herein, where components and/or actions/operationsin dashed line may be optional. Although the operations described belowin FIGS. 2-3 are presented in a particular order and/or as beingperformed by an example component, the ordering of the actions and thecomponents performing the actions may be varied, in some examples,depending on the implementation. Moreover, in some examples, one or moreof the following actions, functions, and/or described components may beperformed by a specially-programmed processor, a processor executingspecially-programmed software or computer-readable media, or by anyother combination of a hardware component and/or a software componentcapable of performing the described actions or functions.

FIG. 1 is a schematic diagram of an example of a wireless communicationsystem 100 that includes one or more networks, such as network 1 102,having one or more service event loggers 104 for logging service eventsoccurring on resources of the network 1 102. For example, the resourcesof the network 1 102 may include various types of nodes, such ascomputing devices, databases, devices with a network-specificfunctionality, such as routers, bridges, firewalls, web servers, loadbalancers, etc., and/or the like. Each resource may have an associatedservice event logger 104 to log service events in a service eventrepository 106, where the service event logger 104 may operate on theresource or otherwise to detect communications from the resource forlogging the service events. In an example, the service events in serviceevent repository 106 may include various types of events to notify of ahealth or status of one or more resources, such as processor or memoryutilization on the resource, throughput of traffic on the resource,application-specific events that are definable by applications executingon the resource, etc. The service events may also include or be referredto as incident reports to identify certain incidents occurring onresources. In one example, an incident report can include an incidentderived from multiple service events detected with one or more definedparameter values (e.g., a poor or non-existent connection for a networkresource based on detecting one or more consecutive service eventsrelated to a dropped connection).

A computing device 110 can be provided that can execute a networkdiagnostic application 112 to obtain service events from the serviceevent repository 106 for inspection thereof and/or taking remedial stepsto resolve an identified incident. As described, this can result in avast number of service events being generated and stored in the serviceevent repository 106 over a short period of time, and as such monitoringeach service event can become overwhelming and ineffective fordiagnosing possible issues in the network.

For example, another computing device 120 is provided for exposing aframework to obtain service event information from service eventrepository 106 and for generating additional structures to assist inprocessing the vast number of service events in the service eventrepository in accordance with aspects described herein. For example,computing device 120 can include or can otherwise be coupled with aprocessor 124 and/or memory 126, where the processor 124 and/or memory126 can be configured to execute or store instructions or otherparameters related to processing service events, generating amultiple-layer relational graph defining relationships among the serviceevents, responding to queries for service events, etc., as describedherein. For example, processor 124 and memory 126 may be separatecomponents communicatively coupled by a bus (e.g., on a motherboard orother portion of a computing device, on an integrated circuit, such as asystem on a chip (SoC), etc.), components integrated within one another(e.g., processor 124 can include the memory 126 as an on-board component121), and/or the like. Memory 126 may store instructions, parameters,data structures, etc., for use/execution by processor 124 to performfunctions described herein.

In an example, computing device 120 can execute an operating system 128(e.g., via processor 124 and/or memory 126) for providing an environmentfor executing one or more components or applications, such as a networkdiagnostic component 130 for fulfilling requests for service event datafrom the service event repository 106, as requested by networkdiagnostic application(s) 112 on one or more other computing devices110, a graphing component 132 for generating a multiple-layer relationalgraph 150 defining multiple layers of relationships between serviceevents in the service event repository 106, and/or a query processingcomponent 134 for processing a query context for a service event basedon determining one or more related service events from themultiple-layer relational graph 150.

In an example, graphing component 132 may include a layer generatingcomponent 140 for generating the multiple layers of the multiple-layerrelational graph 150. For example, layer generating component 140 caninclude a configuration obtaining component 142 for obtaining aconfiguration (e.g., as stored in memory 126 or other memory or fromanother device related to the network, another device for configuringnetwork diagnostic analysis, etc.) where the configuration can specifyrelationships between service events or corresponding services, andgenerating the configuration layer 152 of the multiple-layer relationalgraph 150 to indicate relationships based on the obtained configuration.In another example, layer generating component 140 can include anobserving component 144 for observing network traffic, user behavior ofthe network diagnostic application 112, etc. with respect to the serviceevents and/or corresponding services, and generating the observationlayer 154 of the multiple-layer relational graph 150 to indicaterelationships based on the observations. In another example, layergenerating component 140 can include a learning component 146 forperforming anomaly detection of key services or service events in theservice event repository 106, and generating the learned layer 156 ofthe multiple-layer relational graph 150 to indicate relationships basedon the detected anomalies in the service events.

In one example, query processing component 134 can process querycontexts for service events received by or from the network diagnosticcomponent 130 to provide additional service events that may be ofinterest based on a set of service events or services in the querycontext. For example, query processing component 134 can query graphingcomponent 132 to determine the one or more additional service eventsbased on relationships specified in the multiple-layer relational graph150. The relationships can be identified at each of (or one or more of)the different layers 152, 154, 156. Query processing component 134 candetermine whether to include the additional service events based onwhich layer(s) indicate the relationship and/or different associatedmetrics, such as an observation count in the observation layer 154, aconfidence metric of the relationship in the learned layer 156, etc.

Computing device 110 can also similarly include a processor 124, memory126, operating system 128, etc., for operating the network diagnosticapplication 112 and/or other features or functions described herein.These components are not shown in the computing device 110 in FIG. 1 forease of explanation.

FIG. 2 is a flowchart of an example of a method 200 for determiningrelated service events in processing a query for a set of one or moreservice events. For example, method 200 can be performed by thecomputing device 120, and is accordingly described with reference toFIG. 1, as a non-limiting example of an environment for carrying outmethod 200.

In method 200, at action 202, a query context for service eventsoccurring on a network can be received. In an example, query processingcomponent 134, e.g., in conjunction with processor 124, memory 126,operating system 128, etc., can receive the query context for serviceevents occurring on the network. For example, query processing component134 can receive the query context from network diagnostic component 130,where the network diagnostic component 130 can receive a correspondingquery from a network diagnostic application 112 executing on anothercomputing device 110. For example, network diagnostic component 130 canfacilitate querying of service events in service event repository 106,as described, and can provide various network diagnostic applications112 with service event information (e.g., incident reports, etc.) basedon a request from a network diagnostic application 112, based on asubscription from the network diagnostic application 112 to receivecertain service events (e.g., for certain resources and/or for certaintypes of service events, etc.), and/or the like.

In one specific example, a query context can relate to a signal that canrepresent service events, such as a signal line representing processorutilization at a network resource. In this example, network diagnosticapplication 112 may request service events related to the processorutilization at the network resource, which may include periodic serviceevents received from the network resource (e.g., via a service eventlogger 104) that report the processor utilization. Network diagnosticapplication 112 can utilize the service events to generate a signal on auser interface representing the processor utilization reported in theservice events. Examples are shown in FIGS. 4 and 5, which are describedin further detail below.

In one example, network diagnostic component 130 can also implementsecurity policies that define security contexts for users to accesscertain service events for certain nodes, certain types of serviceevents, etc. In this example, network diagnostic component 130 canensure a network diagnostic application 112 has the security clearance(e.g., a user of the application 112 is in a certain security group) toreceive the requested service event information. In any case, networkdiagnostic component 130 can provide requested service event informationto the corresponding network diagnostic application 112.

In method 200, at action 204, a set of service events occurring in thenetwork can be determined based on the query context. In an example,query processing component 134, e.g., in conjunction with processor 124,memory 126, operating system 128, etc., can determine, based on thequery context, the set of (e.g., one or more) service events occurringin the network. For example, the request can be a request/responserequest, a subscription request, etc., that can indicate one or moreparameters in the query context. The one or more parameters may identifya type of service event, a service, a corresponding network resource,and/or the like. In one example, the query content may indicate a useraccount for the network diagnostic application 112, a network resourceor machine being viewed, and/or a view (or related parameters) ofservice events for the network resource. Given this information, forexample, network diagnostic component 130 can obtain data (e.g.,including a collection of one or more service events) from service eventrepository 106 for providing to the corresponding network diagnosticapplication 112.

For example, network diagnostic component 130 can query the serviceevent repository 106 to obtain the service events as requested (e.g., asrelated to a certain service and/or network resource). In anotherexample, network diagnostic component 130 can receive (e.g., based on asubscription) service events from the service event repository 106 thatcorrespond to the certain service and/or network resource (e.g., wherethe service event repository can callback a callback function of thenetwork diagnostic component 130 based on receiving the service eventsand associating the service events with a subscription, etc.).

In addition, e.g., based on the query context, network diagnosticcomponent 130 can additionally query a multiple-layer relational graph150 for additional service events that may be related to the querycontext or the set of service events specified in the query context.Query processing component 134 can obtain the query and can determinethe set of service events occurring in the network based on the querycontext (e.g., similarly as network diagnostic component 130 and/or canreceive this information directly from network diagnostic component130). Given the set of one or more service events in the query context,additional services possibly of interest can be identified from themultiple-layer relational graph, as described herein.

In one example, a query context can relate to a view of the networkdiagnostic application 112 that may be associated with multiple serviceevents over a period of time, such as a signal showing resource usageover a period of time, where the resource usage is indicated over timein various service events. In this example, network diagnostic component130 can query the service event repository to determine the serviceevents indicating the resource usage for the service over the period oftime, and network diagnostic application 112 can graphically representthe usage as a signal line over time. Network diagnostic application 112can be used to generate multiple views in this regard, where each viewcan have an underlying query context for obtaining corresponding serviceevents used to create the views. Thus, as an example of correlatingservice events based on observation, where views are generated forviewing together, a relationship between the underlying queries may beobserved as occurring at similar times, for similar users, on similarnetwork diagnostic application 112 or computing device 110, etc., asdescribed further herein.

In method 200, at action 206, multiple-layers of a multiple-layerrelational graph can be queried to determine one or more other serviceevents having a defined relationship with the set of service events atone or more of the multiple layers. In an example, query processingcomponent 134, e.g., in conjunction with processor 124, memory 126,operating system 128, etc., can query the multiple layers of themultiple-layer relational graph 150 to determine the one or more otherservice events having a defined relationship with the set of serviceevents at one or more of the multiple layers. For example, queryprocessing component 134 can query the configuration layer 152, theobservation layer 154, and/or the learned layer 156 of themultiple-layer relational graph 150 to determine a relationship betweenthe set of services and the one or more other services at least at oneor more of the layers 152, 154, 156. The related service events may bedetermined as related based on a relation between the underlyingservices, which can be determined from one or more of the layers 152,154, 156, and/or other considerations, such as based on a timing of theservice events (e.g., service events occurring within a threshold timeof one another) of the related services, and/or the like. Though shownas part of the same computing device 120, in an example, queryprocessing component 134 can be at a different computing device thangraphing component 132 that generates, manages, and/or stores themultiple-layer relational graph 150.

For example, given a service event in the set of one or more serviceevents, query processing component 134 may identify a relationship withone or more other service events in the configuration layer 152, theobservation layer 154, and/or the learned layer 156. As described, theconfiguration layer 152 can indicate (e.g., and/or may store anindication of) a relationship between the service event and one or moreother service events as specified in a configuration. For example, theconfiguration may be generated using a user interface to allow a user toindicate known relationships between service events and/or correspondingservices or by another mechanism.

The observation layer 154 can indicate (e.g., and/or may store anindication of) a relationship between the service event and one or moreother service events that is based on observing network traffic ofrequests for the service event (or for a similar type of service event,for other events for the corresponding service, etc.) and similarnetwork traffic (e.g., occurring at a similar time, from a similarcomputing device 110 or user account, etc.) of the one or more otherservice events.

The observation layer 154 can additionally or alternatively indicate(e.g., and/or may store an indication of) a relationship between theservice event and one or more other service events that is based onobserving user activity (e.g., of the network diagnostic application112) in requesting and/or viewing the service event (or similar types ofservice events, other events for the corresponding service, etc.) andthen also requesting and/or viewing the one or more other serviceevents. For each observed relationship, the observation layer 154 mayinclude one or more metrics, in one example, such as an observationcount for the number of times the observed relationship criteria isdetected. For example, the observations can be made in real-time or nearreal-time as traffic or user activity occurs, or can be madeafter-the-fact based on analyzing network traffic logs, logs of useractivity on network diagnostic component 130, etc.

The learned layer 156 can indicate (e.g., and/or may store an indicationof) a relationship between the service event and one or more otherservice events that is based on algorithmic determinations regarding theservice events within the service event repository 106, such as bydetecting data anomalies corresponding to the other service events basedon keying the service event. For each anomaly, the learned layer 156 mayinclude one or more metrics, in one example, such as a confidence metricfor the determined relationship.

In querying the multiple layers at action 206, optionally at action 208,a metric based on the results of querying the multiple layers can bedetermined. In an example, query processing component 134, e.g., inconjunction with processor 124, memory 126, operating system 128, etc.,can determine the metric based on the results of querying the multiplelayers. For example, query processing component 134 can determine themetric based on whether a relationship is determined from a given layerand/or based on the layers within which the relationship exists. Forexample, query processing component 134 can determine a first metricwhere the relationship is determined from the configuration layer 152.In one example, this can be a highest metric and/or can definitivelyidentify a relationship between the service in the set of one or moreservices and the other services, as the relationship can be explicitlyidentified by a user. In addition, for example, the metric can bedetermined based on one or more other metrics observed or obtained fromeach layer, such as an observation count in the observation layer for anobserved relationship between the service events (and/or types ofservice events), confidence score in the learned layer, etc., asdescribed.

Moreover, in determining the metric at action 208, optionally at action210, one or more weights can be applied to a result metric for eachlayer. In an example, query processing component 134, e.g., inconjunction with processor 124, memory 126, operating system 128, etc.,can apply the one or more weights to the result metric for each layer(or one or more of the layers). For example, query processing component134 can apply higher weights to metrics for the configuration layer 152,as described, and/or can determine any desirable weighting for eachlayer. In one example, weighting the metrics for the layers 152, 154,156 may be based on feedback of whether correlations between serviceevents is accurate (e.g., based on being presented via an interface). Inany case, the weights and/or metrics can be compared with threshold(s)to determine whether to indicate a correlation between a determined setof service events and the other service events discovered from themultiple-layer relational graph.

In addition, in an example, query processing component 134 can furtherperform pattern mining or other machine-learning algorithms based on amore limited set of correlated services and/or service events determinedfrom the multiple-layer relational graph 150. In this example, queryprocessing component 134 can further distill a list of services and/orservice events determined as possibly related (e.g., such to indicatethe other service events in reporting the determined service events)from the multiple layers 152, 154, 156 of the graph 150 by performingpattern mining on the list of services and/or service events.

In method 200, at action 212, the one or more other service events canbe indicated via an interface and in response to the query context. Inan example, query processing component 134 and/or network diagnosticcomponent 130, e.g., in conjunction with processor 124, memory 126,operating system 128, etc., can indicate, via the interface (e.g., userinterface) and in response to the query context, the one or more otherservice events. For example, query processing component 134 can indicatethe one or more other service events to the network diagnostic component130 for providing to the corresponding network service application 112requesting the query. In an example, query processing component 134 candetermine whether to indicate the one or more other service events basedon the determined metric and/or can indicate any other service eventsfor which a relationship is identified (or determined to have anassociated metric that achieves a threshold) in one of the multiplelayers of the multiple-layer relational graph 150, in a threshold numberof the multiple layers of the multiple-layer relational graph 150, ineach of the multiple layers of the multiple-layer relational graph 150,etc.

Additionally, for example, query processing component 134 and/or networkdiagnostic component 130 may indicate the one or more other serviceevents including an indication of a relationship to the set of serviceevents determined for the query context. The indication of relationshipmay include an identifier for the other service event(s) indicating therelationship and/or a level of relationship (e.g., a metric, weight,and/or the like, as described).

For example, network service application 112 can provide an indicationof the one or more other service events received from the queryprocessing component 134 or network diagnostic component 130 usingvarious mechanisms. For example, network service application 112 canprovide the indication as another view or signal line representing theone or more other service events presented along with a view that maycorrelate to the query context. In another example, network serviceapplication 112 can provide the indication as a list of the otherservice events, an indication of the other service events occurring attimes corresponding to the set of service events that correlate to thequery context, etc. In yet another example, network service application112 can provide the indication as a pop-up or other notification thatthere are possibly related service events (e.g., the other serviceevents) to the service events that are the subject of the query context.Moreover, as described, the network service application 112 may alsoprovide a mechanism for indicating feedback for the indication of theother service events (e.g., feedback as to whether the other serviceevents are relevant to the service events that are the subject of thequery context).

In method 200, optionally at action 214, feedback indicating whether theone or more other service events are relevant to the set of serviceevents can be received. In an example, query processing component 134and/or network diagnostic component 130, e.g., in conjunction withprocessor 124, memory 126, operating system 128, etc., can receive thefeedback indicating whether the one or more other service events arerelevant to the set of service events. For example, as described,network service application 112 can provide an interface for promptingfor feedback of the relevancy, and can provide any indicated feedback tothe query processing component 134 and/or network diagnostic component130. For example, the feedback can indicate whether the one or moreother service events are relevant to the set of service events that arethe subject of the query context, a level of relevancy, and/or the like.

In method 200, optionally at action 216, one or more layers of themultiple-layer relational graph can be modified based on the feedback.In an example, graphing component 132, e.g., in conjunction withprocessor 124, memory 126, operating system 128, etc., can modify one ormore layers of the multiple-layer relational graph 150 (e.g., theconfiguration layer 152, the observation layer 154, or other layers)based on the feedback. For example, graphing component 132 may modifymetrics associated with observations at the observation layer 154 basedon the feedback (e.g., improve a metric where the feedback is positive,decrease the metric or delete an observation association where thefeedback is negative, etc.).

FIG. 3 is a flowchart of an example of a method 300 for generating amultiple-layer relational graph indicating relationships between serviceevents and/or corresponding services. For example, method 300 can beperformed by the computing device 120, and is accordingly described withreference to FIG. 1, as a non-limiting example of an environment forcarrying out method 300. In addition, method 300 can be performed inpreparation for fulfilling queries and/or determining related serviceevents, as described in method 200. In another example, method 300 canbe performed as a real-time or near real-time process as part ofquerying the multiple-layer relational graph at action 206 of method200.

In method 300, at action 302, a configuration layer of a multiple-layerrelational graph can be generated based on relationships betweenservices as defined in a stored configuration. In an example, layergenerating component 140, e.g., in conjunction with processor 124,memory 126, operating system 128, graphing component 132, etc., cangenerate the configuration layer of the multiple-layer relational graphbased on relationships between services as defined in the storedconfiguration. For example, configuration obtaining component 142 canobtain the stored configuration (e.g., from memory 126 and/or fromanother computing device, etc.), which can be generated based on userinput received via an interface for defining relationships betweenservice events and/or between corresponding services. As described,services may depend on one another, and this dependency can be indicatedin the stored configuration. This can allow for determining arelationship between service events occurring on the dependent services(e.g., at a similar time or otherwise).

In one example, the configuration can define a relationship betweenservice events based on collating and linking of underlying incidentrecords by on-call engineers with incident management and serviceobservability systems. For example, a user of network diagnosticapplication(s) 112 executing on various computing devices 110 canindicate the linking of the incident records and/or service events viaan interface option on the network diagnostic application 112. In otherexamples, other applications can be used to indicate the configuredassociations between service events, service event types, services,incident reports, incident report types, etc. The configuration layer152 may include an indication of a relationship (or link) between atleast a subject service and the one or more other services, such thatthe query processing component 134 can identify the link and report theother services or service events of the other services (e.g., occurringat a similar time or otherwise indicated as depending on the subjectservice event) as possibly of interest.

In method 300, at action 304, an observation layer of the multiple-layerrelational graph can be generated based on relationships betweenservices based on monitoring network traffic or observing user behavior.In an example, layer generating component 140, e.g., in conjunction withprocessor 124, memory 126, operating system 128, graphing component 132,etc., can generate the observation layer of the multiple-layerrelational graph based on relationships between services based onmonitoring network traffic or other topological relationships orobserving user behavior.

For example, observing component 144 can monitor the network traffic(e.g., coming from network diagnostic application(s) 112 or networkdiagnostic component 130) to determine correlated requests for servicesor service events. For example, where observing component 144 observessimilar patterns in requests for services and/or service events atdifferent times based on the network traffic, whether from the samenetwork diagnostic application(s) 112 or different network diagnosticapplication(s), or other topological relationships between signalsources (e.g., the source being the service from which the service eventis logged), observing component 144 may infer an observed relationshipbetween the services and/or service events.

Similarly, where observing component 144 observes similar patterns inrequests for services and/or service events at different times based onuser behavior on the network diagnostic application 112 (e.g., asobserved from the network diagnostic application 112 itself or requestsreceived at the network diagnostic component 130), observing component144 may infer an observed relationship between the services and/orservice events. In one example, observing component 144 can observe userbehavior of the diagnostic application 112 itself, which in one specificexample may include a configuration of a user-defined interface of thenetwork diagnostic application 112. For example, a user may define auser interface to analyze health or other metrics of network resources,where the interface may display signals generated based on observedservice events (e.g., service events reporting resource utilization). Inone specific example, based on physical proximity of signals on theinterface (e.g., as being next to one another, part of the samechart/graph, etc.), observing component 144 can determine a relationshipbetween the corresponding services. The information regarding theuser-defined interface layout may be provided to the network diagnosticcomponent 130, from which the observation layer 154 can receive suchinformation.

An example is shown in FIG. 4, which illustrates an example of a userinterface 400 of a network diagnostic application 112. In user interface400, a user thereof may have defined the user interface 400 to includesignals 402, 404, 406, 408 in the view. The signals 402, 404, 406, 408may each correspond to a set of service events for different servicesthat the user desires to monitor. The signals 402, 404, 406, 408 mayshow information of the service events (e.g., reliability, incomingrequest rate or reliability, etc.) over the same or similar period oftime and/or at the same or similar time instances. In this example,observing component 144 can determine that the user interface 400includes the signals 402, 404, 406, 408 on the same view and/or within athreshold physical proximity within the view, that the user interface400 processes interactions on the signals 402, 404, 406, 408 at similarpoints in time, etc., and can accordingly observe a relationship betweenthe corresponding service events and/or underlying services, which canbe set in the observation layer 154 for subsequently determining relatedservices or service events. As described, for example, observingcomponent 144 can observe such properties of the user interface 400based on at least one of determining the user interface 400 defined onthe network diagnostic component 130 that facilitates operation of thenetwork diagnostic application 112, receiving, at the network diagnosticcomponent 130, an alert of creation/modification of the user interface400 on the network diagnostic application 112, and/or the like.

In any case, the observation layer 154 may include an indication of arelationship (or link) between at least a subject service and the one ormore other services, such that the query processing component 134 canidentify the link and report the other services or service events of theother services (e.g., occurring at a similar time or otherwise indicatedas depending on the subject service event) as possibly of interest.

In one example, observing component 144 may include an observationcount, frequency, etc. based on a number of observations of the servicesand/or service events within a period of time, where the observationcount may indicate a likelihood of correlation between the servicesand/or service events. Thus, for a given service or service event, queryprocessing component 134 can determine related services or serviceevents based on the observations, observation count, etc., to provide inresponse to a query for the given service. For example, theseobservations can indicate what services and/or service events on-callengineers are looking at when looking at the given service or serviceevent, as described.

In method 300, at action 306, a learned layer of the multiple-layerrelational graph can be generated based on relationships betweenservices based on performing anomaly detection on key services. In anexample, layer generating component 140, e.g., in conjunction withprocessor 124, memory 126, operating system 128, graphing component 132,etc., can generate the learned layer of the multiple-layer relationalgraph based on relationships between services based on performinganomaly detection on key services (e.g., a subject service wheregenerating the learned layer 156 is performed in real-time or nearreal-time or otherwise). For example, learning component 146 can performcorrelations, anomaly detection, or other machine-learning algorithms(e.g., pattern mining) on the services and/or service events in theservice event repository 106 to identify likely related services and/orservice events.

The learned layer 156 may include an indication of a relationship (orlink) between at least a subject service and the one or more otherservices, such that the query processing component 134 can identify thelink and report the other services or service events of the otherservices (e.g., occurring at a similar time or otherwise indicated asdepending on the subject service event) as possibly of interest. Forexample, the learned layer 156 can detect anomalies in certain serviceevent data over a period of time, such as resource utilization ofservices or related network nodes based on reported service events. Forexample, anomalies can be detected in similar changes in utilizationamounts, the times at which utilization changes (e.g., regardless ofwhether the amount is similar), etc.

In one example, learning component 146 may determine a confidence scoreor other metric for identified anomalies between services and/or serviceevents, which can be included in the learned layer 156. Thus, for agiven service or service event, query processing component 134 candetermine related services or service events based on the detectedanomalies, the confidence score or other metric, etc., to provide inresponse to a query for the given service. In one example, theconfidence score may be based on a number of correlations observedbetween the potentially related services or service events.

An example is shown in FIG. 5, which illustrates an example of agraphical depiction of signals 500 related to service events, wheresignal 502 relates to a set of service events of a service, such asresource utilization, etc., as described, and signal 504 relates to adifferent set of service events that may be determined as related to theset of service events of signal 502 based on correlation or othermachine-learning algorithms. For example, correlation may show eventshappening at similar time instances, indicated by symbols 506. In anexample, learning component 146 may determine a relationship between theunderlying service events based on detecting a threshold number ofevents happening in each signal within a period of time (and/or aconfidence score may be computed based on the frequency of correlatedevents among the signals or underlying service events). In an example,learning component 146 can set a determined relationship and/or relatedmetrics in the learned layer 156 for subsequently determining relatedservices or service events.

In the multiple-layer relational graph 150, relational data from thevarious layers 152, 154, 156 can be combined, as described, and used tobuild a knowledge graph between the services and their metrics.Traversal of this graph 150 can be useful in various applications, suchas root cause analysis, determining most failing metrics, grouping ofrelated metric failures etc.

FIG. 6 illustrates an example of a relational graph representation 600of a set of service events. For example, representation 600 can indicatea signal of interest, which can refer to a metric (e.g., processorusage) measured on a specific network resource based on a collection ofservice events (e.g., events that indicate resource usage at periods oftime). For example, the signal of interest can be requested and/ordefined by a network diagnostic application 112 on a computing device110 to receive, from the service event repository 106 in real-time, nearreal-time or otherwise, processor usage service events for the networkresource via network diagnostic component 130. In an example, a querycontext can include an indication of the user, the service or associatednetwork resource, and a view of the network diagnostic application 112being requested (which can indicate the desired service events). Indetermining relationships with other service events, query processingcomponent 134 can query the configuration layer 152 to determine thatservice 1 on machine A depends on service 2 on machine B, and/orspecifically that the processor usage on service 1 on machine A dependson incoming API reliability on service 2 on machine B. Thus, queryprocessing component 134 can provide, in response to a query forprocessor usage of service 1 on machine A, corresponding API reliabilityservice events for service 2 on machine B.

Similarly, query processing component 134 can query the observationlayer 154 to determine service events typically viewed by this user (orother users) along with the service event that is subject of the view todetermine additional service events of interest (and/or related views ofthe additional service events, such as other signals). For example,query processing component 134 can determine a relationship (e.g.,frequency looked at) indicated on observation layer 154 between the userlooking at service 5 on machine F when also looking at service 1 onmachine A. As described, this observation may be determined based on auser-defined interface that includes views of metrics for service 5 onmachine F and service 1 on machine A (and specifically for outgoingrequest rate for service 5 on machine F with the processor usage timefor service 1 on machine A). Thus, query processing component 134 canprovide, in response to a query for processor usage of service 1 onmachine A, corresponding outgoing request rate events for service 5 onmachine F.

In another example, query processing component 134 can determine arelationship (e.g., frequency seen with) indicated on observation layer154 between network traffic for obtaining metrics related to service 4at machine D around the same time or times as network traffic forobtaining metrics related to service 1 on machine A (and specificallyfor outgoing request rate for service 4 on machine D with the processorusage time for service 1 on machine A). Thus, query processing component134 can provide, in response to a query for processor usage of service 1on machine A, corresponding outgoing request rate events for service 4on machine D.

Similarly, query processing component 134 can query the learned layer156 to determine service events that are historically anomalous with theservice event that is subject of the view to determine additionalservice events of interest (and/or related views of the additionalservice events, such as other signals). For example, query processingcomponent 134 can determine a relationship indicated on learned layer156 between service 3 on machine E (and specifically outgoing requestrate) and at service 1 on machine A. Thus, query processing component134 can provide, in response to a query for processor usage of service 1on machine A, corresponding outgoing request rate events for service 3on machine E.

In any case, for example, the network diagnostic application 112 canindicate potential relationship between the various service events basedon the correlations that are detected/observed at each or one or more(or all) layers in the multiple-layer relational graph.

FIG. 7 illustrates an example of computing device 120 includingadditional optional component details as those shown in FIG. 1. In oneexample, computing device 120 may include processor 124 for carrying outprocessing functions associated with one or more of components andfunctions described herein. Processor 124 can include a single ormultiple set of processors or multi-core processors. Moreover, processor124 can be implemented as an integrated processing system and/or adistributed processing system.

Computing device 120 may further include memory 126, such as for storinglocal versions of applications being executed by processor 124, relatedinstructions, parameters, etc. Memory 126 can include a type of memoryusable by a computer, such as random access memory (RAM), read onlymemory (ROM), tapes, magnetic discs, optical discs, volatile memory,non-volatile memory, and any combination thereof. Additionally,processor 124 and memory 126 may include and execute an operating systemexecuting on processor 124, one or more applications, such as a networkdiagnostic application/component 112/130, graphing component 132, queryprocessing component 134, and/or components thereof, as describedherein, and/or other components of the computing device 120.

Further, computing device 120 may include a communications component 702that provides for establishing and maintaining communications with oneor more other devices, parties, entities, etc. utilizing hardware,software, and services as described herein. Communications component 702may carry communications between components on computing device 120, aswell as between computing device 120 and external devices, such asdevices located across a communications network and/or devices seriallyor locally connected to computing device 120. For example,communications component 702 may include one or more buses, and mayfurther include transmit chain components and receive chain componentsassociated with a wireless or wired transmitter and receiver,respectively, operable for interfacing with external devices. Forexample, communications component 702 can carry communications between anetwork diagnostic application/component 112/130, graphing component132, query processing component 134, etc. executing on another device(or the same device), etc., as described in various examples herein.

Additionally, computing device 120 may include a data store 704, whichcan be any suitable combination of hardware and/or software, thatprovides for mass storage of information, databases, and programsemployed in connection with examples described herein. For example, datastore 704 may be or may include a data repository for applicationsand/or related parameters not currently being executed by processor 124,may include the service event repository 106, etc. In addition, datastore 704 may be a data repository for an operating system, application,such as a network diagnostic application/component 112/130, graphingcomponent 132, query processing component 134, and/or componentsthereof, etc. executing on the processor 124, and/or one or more othercomponents of the computing device 120.

Computing device 120 may also include a user interface component 706operable to receive inputs from a user of computing device 120 andfurther operable to generate outputs for presentation to the user (e.g.,via a display interface to a display device). User interface component706 may include one or more input devices, including but not limited toa keyboard, a number pad, a mouse, a touch-sensitive display, anavigation key, a function key, a microphone, a voice recognitioncomponent, a gesture recognition component, a depth sensor, a gazetracking sensor, any other mechanism capable of receiving an input froma user, or any combination thereof. Further, user interface component706 may include one or more output devices, including but not limited toa display interface, a speaker, a haptic feedback mechanism, a printer,any other mechanism capable of presenting an output to a user, or anycombination thereof.

Computing device 120 can also include a network diagnosticapplication/component 112/130 for generating a query context related toone or more service events, a graphing component 132 for generating amultiple-layer relational graph defining relationships between serviceevents, and/or a query processing component 134 for processing queriesfor service events by providing one or more other service events basedon relationships defined in the multiple-layer relational graph, asdescribed herein.

By way of example, an element, or any portion of an element, or anycombination of elements may be implemented with a “processing system”that includes one or more processors. Examples of processors includemicroprocessors, microcontrollers, digital signal processors (DSPs),field programmable gate arrays (FPGAs), programmable logic devices(PLDs), state machines, gated logic, discrete hardware circuits, andother suitable hardware configured to perform the various functionalitydescribed throughout this disclosure. One or more processors in theprocessing system may execute software. Software shall be construedbroadly to mean instructions, instruction sets, code, code segments,program code, programs, subprograms, software modules, applications,software applications, software packages, routines, subroutines,objects, executables, threads of execution, procedures, functions, etc.,whether referred to as software, firmware, middleware, microcode,hardware description language, or otherwise.

Accordingly, in one or more examples, one or more of the functionsdescribed may be implemented in hardware, software, firmware, or anycombination thereof. If implemented in software, the functions may bestored on or encoded as one or more instructions or code on acomputer-readable medium. Computer-readable media includes computerstorage media. Storage media may be any available media that can beaccessed by a computer. By way of example, and not limitation, suchcomputer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that can be used to carry or store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer. Disk and disc, as used herein, includescompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), and floppy disk where disks usually reproduce data magnetically,while discs reproduce data optically with lasers. Combinations of theabove should also be included within the scope of computer-readablemedia.

The previous description is provided to enable any person skilled in theart to practice the various examples described herein. Variousmodifications to these examples will be readily apparent to thoseskilled in the art, and the generic principles defined herein may beapplied to other examples. Thus, the claims are not intended to belimited to the examples shown herein, but is to be accorded the fullscope consistent with the language claims, wherein reference to anelement in the singular is not intended to mean “one and only one”unless specifically so stated, but rather “one or more.” Unlessspecifically stated otherwise, the term “some” refers to one or more.All structural and functional equivalents to the elements of the variousexamples described herein that are known or later come to be known tothose of ordinary skill in the art are intended to be encompassed by theclaims. Moreover, nothing disclosed herein is intended to be dedicatedto the public regardless of whether such disclosure is explicitlyrecited in the claims. No claim element is to be construed as a meansplus function unless the element is expressly recited using the phrase“means for.”

What is claimed is:
 1. A computer-implemented method for detectingrelated service events in a network diagnostic application, comprising:receiving a query context for service events occurring on one or morenetworks; determining, based on the query context, a set of serviceevents occurring on the one or more networks; querying multiple layersof a multiple-layer relational graph to determine one or more otherservice events having a defined relationship with the set of serviceevents at one or more of the multiple layers, wherein the multiplelayers include a configuration layer defining configured relationshipsbetween services or service events, an observation layer definingobserved relationships between services or service events, and learnedlayer defining algorithmically-determined relationships between servicesor service events; and indicating, via a user interface and in responseto the query context, the one or more other service events.
 2. Thecomputer-implemented method of claim 1, further comprising generatingthe configuration layer based on relationships between services asdefined in a stored configuration.
 3. The computer-implemented method ofclaim 1, further comprising generating the observation layer based onrelationships between services determined based on monitoring networktraffic to detect services communicating with one another.
 4. Thecomputer-implemented method of claim 1, further comprising generatingthe observation layer based on relationships between services determinedbased on monitoring user behavior to detect services observed indiagnosing another service.
 5. The computer-implemented method of claim1, further comprising generating the learned layer based onrelationships between services determined based on performing anomalydetection on key services.
 6. The computer-implemented method of claim1, further comprising determining a metric based on results of queryingthe multiple layers, wherein indicating the other service events isbased at least in part on the metric.
 7. The computer-implemented methodof claim 6, wherein determining the metric is based on applying one ormore weights to a result metric for each layer.
 8. A computing devicefor detecting related service events in a network diagnosticapplication, comprising: a memory storing one or more parameters orinstructions for identifying related service events from a service eventrepository; and at least one processor coupled to the memory, whereinthe at least one processor is configured to: receive a query context forservice events occurring on one or more networks; determine, based onthe query context, a set of service events occurring on the one or morenetworks; query multiple layers of a multiple-layer relational graph todetermine one or more other service events having a defined relationshipwith the set of service events at one or more of the multiple layers,wherein the multiple layers include a configuration layer definingconfigured relationships between services or service events, anobservation layer defining observed relationships between services orservice events, and learned layer defining algorithmically-determinedrelationships between services or service events; and indicate, via auser interface and in response to the query context, the one or moreother service events.
 9. The computing device of claim 8, wherein the atleast one processor is further configured to generate the configurationlayer based on relationships between services as defined in a storedconfiguration.
 10. The computing device of claim 8, wherein the at leastone processor is further configured to generate the observation layerbased on relationships between services determined based on monitoringnetwork traffic to detect services communicating with one another. 11.The computing device of claim 8, wherein the at least one processor isfurther configured to generate the observation layer based onrelationships between services determined based on monitoring userbehavior to detect services observed in diagnosing another service. 12.The computing device of claim 8, wherein the at least one processor isfurther configured to generate the learned layer based on relationshipsbetween services determined based on performing anomaly detection on keyservices.
 13. The computing device of claim 8, wherein the at least oneprocessor is further configured to determine a metric based on resultsof querying the multiple layers, wherein the at least one processor isconfigured to indicate the other service events is based at least inpart on the metric.
 14. The computing device of claim 13, wherein the atleast one processor is configured to determine the metric based onapplying one or more weights to a result metric for each layer.
 15. Anon-transitory computer-readable medium, comprising code executable byone or more processors for detecting related service events in a networkdiagnostic application, the code comprising code for: receiving a querycontext for service events occurring on one or more networks;determining, based on the query context, a set of service eventsoccurring on the one or more networks; querying multiple layers of amultiple-layer relational graph to determine one or more other serviceevents having a defined relationship with the set of service events atone or more of the multiple layers, wherein the multiple layers includea configuration layer defining configured relationships between servicesor service events, an observation layer defining observed relationshipsbetween services or service events, and learned layer definingalgorithmically-determined relationships between services or serviceevents; and indicating, via a user interface and in response to thequery context, the one or more other service events.
 16. Thenon-transitory computer-readable medium of claim 15, further comprisingcode for generating the configuration layer based on relationshipsbetween services as defined in a stored configuration.
 17. Thenon-transitory computer-readable medium of claim 15, further comprisingcode for generating the observation layer based on relationships betweenservices determined based on monitoring network traffic to detectservices communicating with one another.
 18. The non-transitorycomputer-readable medium of claim 15, further comprising code forgenerating the observation layer based on relationships between servicesdetermined based on monitoring user behavior to detect services observedin diagnosing another service.
 19. The non-transitory computer-readablemedium of claim 15, further comprising code for generating the learnedlayer based on relationships between services determined based onperforming anomaly detection on key services.
 20. The non-transitorycomputer-readable medium of claim 15, further comprising code fordetermining a metric based on results of querying the multiple layers,wherein indicating the other service events is based at least in part onthe metric.